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Abstract —We study the problem of achieving strong secrecy 
over wiretap channels at negligible cost, in the sense of maintain¬ 
ing the overall communication rate of the same channel without 
secrecy constraints. Specifically, we propose and analyze two 
source-channel coding architectnres, in which secrecy is achieved 
by multiplexing public and confidential messages. In both cases, 
our main contribution is to show that secrecy can be achieved 
withont compromising commnnication rate and by requiring only 
randomness of asymptotically vanishing rate. Our first source- 
channel coding architectnre relies on a modified wiretap channel 
code, in which randomization is performed using the output of 
a source code. In contrast, our second architecture relies on a 
standard wiretap code combined with a modified source code 
termed uniform compression code, in which a small shared secret 
seed is used to enhance the uniformity of the source code output. 
We carry out a detailed analysis of uniform compression codes 
and characterize the optimal size of the shared seed. 


I. Introduction 

While cryptography is traditionally implemented at the ap¬ 
plication layer, physical-layer security aims at ensuring secrecy 
by taking advantage of the inherent noise at the physical- 
layer of communication channels. The benefits of physical- 
layer security are substantiated by numerous theoretical re¬ 
sults [4], [5], in particular those related to the wiretap channel 
model [6], which suggest that one can achieve information- 
theoretic secrecy without sharing secret keys. Although early 
works on physical-layer security were mostly restricted to 
eavesdropping attacks under optimistic assumptions regarding 
channel knowledge, and only established the existence of 
codes for physical-layer security by means of non-constructive 
random coding arguments, there has been much progress 
recently. In particular, attacker models have been extended 
to situations with limited channel knowledge, e.g., with com¬ 
pound channels [7]-[ll], state-dependent channels [12], [13], 
or arbitrarily varying channels [14]-[16]; several explicit low- 
complexity codes with strong information-theoretic secrecy 
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guarantees have also been designed, for instance, based on 
low-density parity check codes [17], polar codes [18]-[20] or 
invertible extractors [21], [22]. 

Despite these recent advances, physical-layer security 
schemes are yet to be integrated into communication systems. 
One factor that may have hindered their adoption is the limited 
attention paid to the cost of physical-layer security, assessed 
in terms of the decrease in achievable communication rates, 
and the additional resources required for its implementation. 
In fact, if one hopes to deploy physical-layer systems, it 
is reasonable to ask that their operation; i) be transparent 
or at least compatible with upper layer protocols, ii) not 
affect communication rates, and iii) not require additional 
resources. However, most studies of physical-layer security 
focus on the characterization of secrecy capacity, which is 
always less than the capacity, thereby suggesting that secrecy 
can only be achieved at the cost of reducing communication 
rates; furthermore, most existing models and coding schemes 
implicitly assume the presence of an unlimited source of 
uniform random numbers to realize a stochastic encoder. 

The objective of this paper is to revisit these assumptions 
and to show that the cost of secrecy can be made negligible, 
i.e., secrecy neither incurs a reduction in overall communica¬ 
tion rate nor requires extra randomness resources. The crux 
of our approach is to analyze the wiretap channel model 
illustrated in Fig. 1 , in which the encoder only uses a random 
seed of vanishing rate. More specifically, the objective is to 
multiplex a confidential source with a public source, while 
maximizing the sum-rate of secret and public communica¬ 
tion. The idea of multiplexing messages to achieve secrecy 
already implicitly appears in the original work of Csiszar 
and Korner [23], and is explicitly formalized in [24]-[26]; 
however, our approach differs in that: (i) we relax the common 
assumption that messages are exactly uniformly distributed, 
which is unrealistic even if messages are compressed with 
optimal source codes [27], [28]; and (ii) we consider a strong 
notion of secrecy. 

The main contributions of this paper are two source-channel 
coding architectures that achieve information-theoretic secrecy 
over this channel model. The first one, illustrated in Fig. 2(a), 
is based on wiretap codes and requires a random seed of 
negligible rate to compress the public source. The second one, 
illustrated in Fig. 2(b), combines a wiretap code designed to 
operate with uniform randomization with a modified source 
encoder, which compresses data while simultaneously ensuring 
near-uniform outputs. This second architecture is slightly more 
restrictive than the first simply because it requires the encoder 
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Fig. 1. Multiplexing of confidential and public sources. The confidential 
source, , must be reconstructible by the receiver and must be kept secret 
from the eavesdropper. The public source, , should be reconstructible by 
the receiver, and information may be leaked to the eavesdropper. 



(a) Architecture based on a modified wiretap code. 



(b) Architecture based on a modified source encoder. 


Fig. 2. Proposed architectures to multiplex confidential source sequences 
and public source sequences . Ud is a uniformly distributed seed, 
whose length d is sub-linear in the code length n. 


and the decoder to share in advance a small secret seed. For 
both architectures the presence of a random seed at the encoder 
is meant to obtain a nearly uniform source from the public 
source, and is thus unnecessary if the public source is uniform. 
Nevertheless, regardless of the architecture, a secret key for 
authentication is required [29], [30]. While both architectures 
achieve the same optimal performance, the former modifies 
the physical layer of the protocol stack whereas the latter 
modifies the application layer, which makes it much easier 
to implement protocol changes. We also highlight that the 
concept of uniform compression introduced and studied in 
Section IV is of independent interest, as it can be used 
in other security problems. For instance, in secure network 
coding [31]-[34], security is typically obtained by injecting 
uniformly distributed “packets” into the network, which the 
destination nodes are able to decode along with the messages. 
Similar to the compression of the public source with uniform 
compression codes in Section IV, these uniformly distributed 
“packets” in secure network coding could be replaced by 
uniformly compressed public messages. 

Our model initially presented in [1] is closely related to the 
concurrent study [35] and the subsequent study [36], with jour¬ 
nal versions [37], [38]. However, our model is not subsumed 


by any of the models considered in [35], [37] or in [36], [38]. 
The main difference with [36], [38] is that we only allow 
a vanishing rate of randomness to be used at the encoder 
to account for all the resources required to achieve strong 
secrecy. This assumption results in an additional constraint on 
the rate of the public source, which is not accounted for by the 
analysis of [36], [38]. We provide additional details on how our 
achievability schemes differ from [36], [38] in Remark 1. Our 
model also differs from [35], [37], as we consider non-uniform 
sources instead of uniform messages, so the analysis in [35], 
[37] does not apply. We further detail in Remark 2 how our 
achievability schemes differ from those in [35], [37]. Because 
of differences in the models considered, the two achievability 
arguments we present are conceptually different from those 
in [35]-[38], and shed a different light on how to implement 
multiplexing. 

Remark 1. In [36], [38], the authors analyze the trans¬ 
mission over a wiretap channel of a common message Sq 
and multiple confidential messages Si,, St that may not 
be jointly independent. Moreover, the encoder is allowed to 
encode these T -\- 1 messages using a randomized encoder. 
In our approach, we have two independent sources, which 
when compressed losslessly but separately, yield two separate 
non-uniform messages. One of these sources is confidential, 
while the other is public and can possibly be leaked to the 
eavesdropper. However, in the two architectures considered 
in our work, the encoding is only allowed to use a random 
seed whose length grows sub-linearly in the code length n. 
This introduces a new constraint on the minimum rate of the 
public source that is absent in [36], [38]. Furthermore, the 
randomized encoding in [36], [38] uses a commutative group 
structure, while our two achievability schemes use either (i) 
typicality-based compression arguments to show that the Renyi 
entropy of order 2 of the compressed public source approaches 
its entropy (see Section III); or (ii) lossless compression codes 
with near uniform encoder output that require a random seed 
whose length grows as 0{i/n), where n is the code length 
(see Section IV). 

Remark 2. In [35], [37], the authors study the broadcast 
channel with confidential messages and precisely analyze 
the trade-offs among the rates of uniform secret messages, 
uniform public messages, and uniform local randomness. In 
contrast, we study a source setting in which non-uniform 
confidential and public sources are transmitted over a wiretap 
channel. We present two distinct achievability arguments in 
Section III and Section IV for the proposed generalization 
that do not naturally follow from the proof arguments in [35], 
[37]. Despite similarities with the converse for our model, the 
converse in [35], [37] does not directly apply to the setting 
considered in Section IV because of the presence of a shared 
seed. Therefore, for completeness a converse for our model is 
provided in Appendix A. 

The remainder of the paper is organized as follows. In 
Section II, we formally describe the communication model 
under consideration. In Sections III and IV, we prove that 
the two architectures shown in Fig. 2 achieve near-optimal 
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performance, i.e., offer the same rate trade-offs as the com¬ 
munication problem without security constraints. More specif¬ 
ically, we show in Section III the existence of wiretap codes 
that ensure secrecy with non-uniform randomization, while in 
Section IV, we show how to render the output of a source 
code nearly uniform. Section V concludes the paper with some 
perspectives for future work. 

II. Preliminaries and Problem Statement 

A. Notation 

Random variables, e.g., X, and their realizations, e.g., x, are 
denoted by uppercase and lowercase serif font, respectively, 
while alphabets, e.g., X, are denoted by calligraphic font. 
Unless otherwise specified, random variables have finite alpha¬ 
bets, and the generic probability mass function of X is denoted 
by px- Basic information-theoretic quantities, e.g., H{X), 
I(X;Y) are dehned as in [39]. For two random variables X 
and X' over the alphabet X, the variational distance between 
X and X' is Y {px,Px') = \Pxix) - px>ix)\ . For 

any e > 0, S{e) denotes a positive function of e such that 
limj^o <5(e) = 0. We also dehne |a, bj = [[aj, |"h]] fl N. 

B. Wiretap channel model 

Let X, y and Z be finite alphabets. As illustrated in 
Fig. 1, we consider a Discrete Memoryless Channel (DMC) 
{X,pYz\x^y X 2,). The channel {X,pY\x^y) is the main 
channel while the channel {X,pz\x^ -2) is the eavesdropper’s 
channel. We assume that the transmitter Alice wishes to trans¬ 
mit the realizations of two independent Discrete Memoryless 
Sources (DMSs) {VcPVa) ™tl {Vp^pv^,)- Both sources are to 
be reconstructed without errors by the receiver Bob observing 
y", while the source (VcPvJ should be kept secret from the 
eavesdropper Eve observing Z”. Hence, we refer to (VcPVc) 
as the confidential source and to (V^,pvp) as the public source. 

Definition 1. A code for Cn the wiretap channel consists of 
the following. 

• A deterministic encoding function /„ : V" x Vp x 
|l,2'^"J —> X^, which maps n symbols of the confi¬ 
dential source and n symbols of the public source to 
a codeword of length n with the help of a uniformly 
distributed seed of length dn bits; 

• A decoding function > V” x Vp, which maps a 

sequence of n channel output observations to n symbols 
of the confidential source and n symbols of the public 
source. 

The performance of is measured in terms of the average 
probability of error 

Pe(C„) 4 P [(!/", 

and in terms of the secrecy metric 
S(C„) = max 

Note that since we do not know the exact output distribution 
of the source encoders, we impose a security constraint akin 


to semantic security [22]. We also require the length of the 
uniformly distributed seed to be sub-linear in n, i.e., 

lim — = 0. 

n—foo TL 

Note that in our second architecture presented in Section IV 
and depicted in Figure 2(b), we allow the seed to be shared 
between the encoder and the decoder, in which case, the seed 
is also an argument to the decoding function p„. 

C. Source-channel coding theorem 

Theorem 1. Consider a confidential DMS {Vc,pv„) t^tid a 
public DMS (VpjPVp) to be transmitted over a wiretap channel 
{X ,py z\Xiy X -2). For any random variable Q over a finite 
alphabet Q such that Q — X — YZ, if 

H{V,) + H{Vp)<I{X-Y\Q) 
H{V,)<I{X-Y\Q)-I{X-Z\Q) , 
H{Vp)>I{X-Z\Q) 

then there exists a sequence of codes such that 

lim Pe(C„) = lim S(C„) = 0. (1) 

n—¥oo n—voo 

Conversely, if there exists a sequence of codes such 

that (1) holds, then there must exist a random variable Q over 
Q with |Q|^ 3 such that Q — X — YZ and 

H{V,) + H{Vp) ^ I{X-,Y\Q) 
H{V,)^IiX-,Y\Q)-IiX-,Z\Q) . 
H{Vp)^I{X-Z\Q) 

Although the result might seem intuitive, the achievability 
proof does not follow from standard arguments and known 
results because of the use of vanishing-rate randomness at 
the encoder. The main contributions of this paper are the two 
achievability proofs detailed next, the first one in Section III 
using the architecture of Fig. 2(a), the second one in Section IV 
using the architecture of Fig. 2(b). Note that the converse 
in [37] does not directly apply to the setting of Section IV, 
because of the presence of a pre-shared seed. We provide a 
detailed proof for the converse of Theorem 1 in Appendix A. 

Remark 3. Unlike the capacity region of the broadcast chan¬ 
nel with confidential messages, the information constraints in 
Theorem 1 do not include an auxiliary random variable V 
such that Q — V — X — YZ. This result is not surprising, 
as this extra random variable accounts for the addition of 
artificial noise (channel prefixing) in the encoder, which is not 
allowed by our model, as we require all encoder inputs to be 
decoded at the receiver. The random variable Q is merely 
a timesharing random variable [1 ], [37]. Similar to [40, 
Appendix C], it is sufficient to consider an alphabet Q such 
that |Q|^ 3 by Fenchel-Eggleston-Caratheodory theorem. 

III. Coding architecture based on wiretap codes 

WITH NON-UNIEORM RANDOMIZATION 

If one were to rely on known wiretap codes [6], [23] to 
transmit the conhdential and public sources, and meet the 
strong secrecy constraint for the confidential source, one would 
have to ensure that the randomization of the encoder could be 
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Fig. 3. Wiretap channel model with non-uniform randomization. 


performed with a nearly uniform source of random numbers, 
measured at least in terms of total variation. If no reconstruc¬ 
tion constraints were placed on the public source {Vp,pvp), 
a natural approach would simply be to extract the intrinsic 
randomness of the source [41] to generate nearly uniform 
random numbers; this strategy happens to be optimal as shown 
in [1, Proposition 1]. However, unlike the model in [1], the 
present setting requires the reconstruction of the public source 
at the receiver. Although lossless compression of the public 
source might intuitively seem to solve the problem, it would 
actually not lead to a uniform random number. As alluded to 
earlier, [27] shows that lossless compression of a source at 
the optimal rate does not necessarily ensure uniformity under 
variational distance. In addition, for DMSs, [28, Theorem 4] 
shows that there exists a fundamental trade-off between re¬ 
construction error probability and uniformity of the encoder 
output measured in variational distance. To circumvent this 
limitation, we design wiretap codes that operate with a non- 
uniform randomization. 


A. Wiretap codes with non-uniform randomization 

We start by studying the wiretap channel model illustrated 
in Fig. 3, in which the objective is to encode a secret 
message G |1,2"^=] by means of a public message 
Mp e 11,2”^^’] ; we do not assume that messages are uniform, 
but we assume that the statistics of the public message Mp 
are known to the encoder. We call the corresponding wiretap 
code a (2"-R‘=, 2”^^’ , n) wiretap code. In this case, we show 
that secrecy is still achievable, but at a rate ^H 2 {Mp), where 
H 2 {Mp) denotes the Renyi entropy of order 2 and is given by 


H2{Mp) 4 


log 


PMpimf 


Proposition 1. Let pqxyz be a joint distribution that factor¬ 
izes as PqPx\qPyz\x- Then, if 

Rc + Rp<I{X-,Y\Q), 

Re<I{X-,Y\Q)-I[X-Z\Q), 

I{X-,Z\Q) < lim ^H 2 {Mp), 

n—foo n 


a sequence of wiretap codes 

lim maxP 

n—>-oo m 

Me ^ Me\Me = m 

= 0, 

lim maxP 

n—>-oo m 

Mp Mp\Mc = m 

= 0, 

lim maxV 

n—foc) m 

fZ^\M„=Tn,PZ^) = 

0. 


Proof: See Appendix B. ■ 

As shown in [1, Proposition 1], if one did not re¬ 
quire the reconstruction of Mp, one could achieve se¬ 
cret rates Rc as in Proposition 1, but with the con¬ 
straint I{X-Z\Q) < lim„_j.oo ^ H{Mp) instead. In general, 
TH 2 {Mp) ^ TH{Mp), and the penalty paid by using the 
Renyi entropy instead of the Shannon entropy may be signif¬ 
icant. The following example highlights an extreme example 
of such a situation. 

Example. Consider Mp G |l,2"^p] such that 

1 _ 

F[Mp = 1] 4 ¥[Mp = z] 4 ^ ifz^l, 

where a sjO, is a parameter that controls the uniformity of 
the distribution. Note that 

lim TH 2 (Mp) = aRp whereas lim Tjjtjyj ^ — r 

n^oo n y y n—>00 n f 

Consequently, the achievable rates predicted in Proposi¬ 
tion 1 could be arbitrarily smaller than those in [1, Proposition 
1 ]. Fortunately, a combination of a source code with a wiretap 
code identihed in Proposition 1 is sufficient to achieve the 
optimal rate of Theorem 1 . 


B. Achievability of Theorem 1 based on wiretap codes with 
non-uniform randomization 

We hrst rehne a known result regarding the existence of 
good source codes. 

Lemma 1. Consider a DMS (V,py). Then, there exists a 
sequence of source encoders /„ : V" x |l, 2'^"] —)■ |l, 2"^"] 
and associated decoders such that 

lim Rp=H{V), lim -H 2 {fn{V'^,Ud„))=Hiy), 

n—>-oo n—>-oo n 

lim P[C" ^ 9„{fniV^,UdJ)] = 0, lim ^ = 0. 

n—>-oo n—>-oo 77, 

Proof: We consider a typical-sequence-based source. 
Specihcally, let n S N, let eg > 0 function of n to be 
determined later, and let TjfiV) be the set of eo-letter- 
typical sequences of length n with respect to pv [39]. The 
typical sequences are labeled with m G |l,2"^"] 

and Rn = ^ log|7^"(C)|. The encoder /„ outputs m if the 
input sequence u" = u"(to) G 71" (V), otherwise it generates 
m G |l,2”^"] uniformly a random. Note that this uniform 
selection when the realization of V" is atypical can be done 
by a random seed Ud„ of appropriate size dp. Decoding 
is performed by returning the typical sequence v"{m) cor¬ 
responding to the received message m. By [39, Theorem 
1.1], we know that P[l/” ^ gn{fn{V" ,Ud„))\ < with 

~ 2|V|e“"'^o''^, Pv — min Pv(f), where supp 
rGsupp(pv) 

denotes the support of a distribution, and i?„ < {l + eo)H{V). 
Hence, for any m G |1,2"^"] 

= F[V" = v"{m) or {V" i Tf^iV) 

and m is drawn uniformly from |l,2”'^"])] 
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\r-{v)\ 

^ ^-n{l-eo)HiV) , ^eoi^) ^_ri(l-eo)H(V) 

^ i-KH 

2-n{l-eo)H{V) 

Hence, since for any discrete random variable X over X, 
H{X) > H 2 {X) > H^{X), we have 


nH{V) > i?(/„(F")) 
^ i?2(/n(F”)) 


= -log(maxp/^(y„)(TO)) 

m V / 

^ n(l - eo)H{V) + log(l - S^g{n)). 


We may choose eg = and ej, > 0. 

Note that the encoder requires Ud^ to encode the non¬ 
typical sequences. To mitigate this requirement, we apply 
the encoder to b{n) sequences of length a(n), where a(n) 
and b{n) are any integers such that a(n)b{n) = n and 
lim„_>.oo a(n) = +oo = lim„_>.oo ^(n).* Hence, the amount 
of required randomness is negligible compared to n since 
P[F“(") <<5,„(a(n)). ■ 

In the remainder of the paper, we refer to the source codes 
identified in Lemma 1 as “typical-sequence based” source 
codes. 

Going back to the setting of Section II-B, let us apply 
Lemma 1 to both sources and (Vp,pvp). Let e > 0. 

There exists Ni G N and two source encoder-decoder pairs, 
denoted and (/^,respectively, such that for n > 

Wi, P[(LJ^,L")^(5-(/-(L")),gP(/P(L;^,,[/dJ))] ^ e. 
We set M, 4 /-(y-) e |l, 2"«=] and Mp ^ UdJ G 

|l, 2”^^’]. Note that we only need randomness for the public 
source. If there exists a distribution pqxyz that satisfies the 
condition of Proposition 1, then, there exists iV 2 G N and a 
wiretap code with encoder-decoder pair {fn,9n) such that for 
n > N 2 , 


maxP 

m 

maxP 

m 


Me ^ Mc|Mc = 771 
Mp ^ Mp\Mc = m 


< e, 

< e, 


maxV (pz"|M<,=m,PZ") < e- 


Encoding the sources into codewords as 
fnif^<y^), fni^p^UdJ), and^ forming estimates from 
the channel output F" as = g'^{gn{Y^)), and 

= 9 ni 9 n{Y'^)), we observe that for n > max{Ni, N 2 ), 


P {V^,V^) ,V^) < 3e and for any u” G V”, 

^ {pz^\v^=v^tPz^) ^ By taking the limit e —?► 0, we 
conclude with Lemma 1 that a code for the wiretap channel 
can be constructed provided 





■ Jf" 


Fig. 4. Source encoder and decoder with uniform outputs. 


IV. Coding architecture based on uniform 

COMPRESSION CODES 

In this section, we develop a second optimal architecture. 
As before, our objective is to circumvent the impossibility of 
generating uniform random numbers with source codes [28, 
Theorem 4], but this time by modifying the operation of the 
source codes themselves. The approach to overcome this im¬ 
possibility is to introduce a small shared uniformly distributed 
random seed. The benefit of this second architecture is that 
it only requires a modification at the application layer of the 
protocol stack. However, the price paid is that the transmitter 
and the receiver must now share a seed whose rate can be 
shown to be made vanishingly small. This contrasts with our 
first architecture in Section III for which the seed is not 
available at the decoder. 

A. Uniform compression codes 

Consider a DMS {X,px)- Let n G N, G N, and 
let Ud„ be a uniform random variable over Ud„ — 
independent of X". In the following we refer to Ud„ as 
the seed and dn as its length. As illustrated in Figure 4, 
our objective is to design a source code to compress and 
reconstruct the DMS [X^px) with the assistance of a seed 
Ud„- 

Definition 2 . A ( 2 "^, n, 2 "^") uniform compression code Cn 
for a DMS {X,px) consists of 

m A message set with M„ = 2"^, 

• A seed set Ud^ = |1, 2“^"], 

• An encoding function fn '■ X” x Udn Min, 

• A decoding function ijjn - Min x X". 

The performance of the code is measured in terms of the 
average probability of error and the uniformity of its output 
as 

Pe(</>n, ^n) = P[-Y” UdJ,UdJ], 

Ue(fn) = V (p<^„(X-,C/<j„),P(7jvr„), 

where Um„ has uniform distribution over Min- 

Remark 4 . Uniformity could be measured with the stronger 
metric U'eifn) = D(P0„(X",c/d„)lbc/M„)> ^Mre D(-||-) is the 
Kullback-Leibler divergence; however, by [42, Lemma 2.7], 
Ue((/'n) can be replaced by if lim nXJeifn) = 0, 

n—>-oo 

which will be the case. 


H{V,) + H{Vp) < I{X-,Y\Q), 

H{V,)<I{X-,Y\Q)-I{X-Z\Q), 

H{Vp)>I{X-Z\Q). 

*A possible choice is a(n) = and b{n) = M with A S]0,1[. 


Definition 3. A rate R is achievable, if there exists a sequence 
o/(2"^, n, 2^^") uniform compression codes {Cn}n^i far the 
DMS {X,px), such that 

lim — logM„ ^ R, lim — = 0, 

n^oo Tl n^oo Jl 
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Fig. 5. Encoding/decoding scheme for Proposition 3. The encoder/decoder 
is obtained from a typical-sequence based source code, and an invertible 
extractor EXTq. 


lim Pe((/)„,-!/'«) = 0, lim lJe{4>n) = 0. 

n— >-oo n— >-oo 

Our main result in this section is the characterization of 
the infimum of achievable rates with uniform compression 
codes as well as the optimal scaling of the seed length d„. 
In the following, we use the Landau notation to characterize 
the limiting behavior of the seed scaling. 

Proposition 2. Let {X,px) be a DMS. The infimum of achiev¬ 
able rates with uniform compression codes is H{X). This 
infimum is achievable with a seed length dn = 0{vn\/n), for 
any {unlngN lini„_).oo Vn = + 00 . Moreover, a necessary 
condition on dnfor a (2”^, n, uniform compression code 
to achieve H{X) is dn = i.e., lim„_>.oo = +oo. 

Proof: See Appendix C. ■ 

B. Explicit uniform compression codes 

As a first attempt to develop a practical scheme for uniform 
compression codes, we propose an achievability scheme for 
Proposition 2 based on invertible extractors [43]. We start by 
recalling known facts about extractors. 

Definition 4 ( [43]). Let e > 0. Let m,d,l € N and let t € 
IR+. A polynomial time probabilistic function Ext ; {0,1}™ x 
{0, I—{0,1}* is called a (m, d, I, t, e)-extractor, if for any 
binary source X satisfying Hao{X) ^ t, we have 

V(pExt(X,C/d),m) < 

where Ud is a sequence of d uniformly distributed bits, 
PUi is the uniform distribution over {0,1}^ Moreover, a 
(to, d, I, t, e)-extractor is said to be invertible if the input can 
be reconstructed from the output and Ud- 

It can be shown [43], [44] that there exist explicit invertible 
(to, d, TO, t, e)-extractors such that 

d = TO — t + 2 logTO + 21og - + 0(1). (2) 

e 

The following proposition shows that one can establish optimal 
uniform compression codes using such invertible extractors. 

Proposition 3. Let {X,px) be a binary memoryless source. 
For any R > H{X) and for any e > 0, the rate R can be 
achieved with a sequence of uniform compression codes such 
that 

• the seed length scales as dn = 0(n^/^"'“'^); 

• the encoder fin '■ 'V" x lAd„ —>■ M.n A composed of 
a typical-sequence based source code combined with an 
invertible extractor as described in Figure 5. 


Proof: See Appendix D. ■ 

Unfortunately, this scheme is not fully practical because it 
relies on a typical-sequence based compression. To provide 
at least one explicit and low-complexity example, we finally 
develop a uniform compression code based on polar codes 
for a binary memoryless source {X,px), X = {0,1}. Let 
e]0,l/2[, n G N, V ^ 2”, and Sx = Let Gn = 

ft ol®" 

be the source polarization transform defined in [45], 

and set = X^Gx- For any set A = of indices 

in |1,V], we define A^[A] = [Ai^, Ai^,..., Ai^^^]. In the 
following, we denote the complement set operation by the 
superscript c. We also define the sets 

Vx = {^ G [1,7V1 : LT > 1 - , 

nx={t&ll,Nl:H{A\A^-^)>Sx}. 

A polar-based uniform compression code is obtained by defin¬ 
ing the encoding function fix as follows. Let denote 

a sequence of uniformly distributed random bits with length 
\Hx\Vx\- Then, 

fixiX^,U\n.\v,\) = [A^[Vx],A^[nx\Vx] 0 C/|«.\Vx|]- 

Proposition 4. Let {X,px) be a binary memoryless source. 
Any rate R > H{X) is achievable with a sequence of polar- 
based uniform compression codes such that the seed length 
\'bLx\^x\ vanishes as the code length grows unbounded. In 
addition, the complexity of the encoding and decoding is 
0{N\ogN), where N denotes the code length. 

Proof: See Appendix E. ■ 


C. Achievability of Theorem 1 based on uniform compression 
codes 

The uniform compression codes of Section IV-A may now 
be combined with known wiretap codes (as depicted in Fig¬ 
ure 2(b)), whose properties we recall in the following lemma. 

Lemma 2 (Adapted from [1, Proposition 1]). Consider a DMC 
[X,pyz\x, y X 2), in which a message Me G |l,2’^^‘'] 

is encoded by means of a uniform auxiliary message Mp G 
|l,2"^J’J. If there exists a joint distribution Pqxyz that 
factorizes as PqPx\qPy z\x ^uch that 

Re + Rp<I{X;Y\Q) (3) 

Re<I{X;Y\Q)-I{X;Z\Q) (4) 

Rp> I{X;Z\Q), (5) 


then there exists a sequence of wiretap codes such 

that 


lim maxP 

n—foo m 

lim maxP 

n—>-oo m 


Me f Mc\Mc = TO 
Mp f=- Mp\Mc = TO 



lim maxV (pz-|M,=m,PZ") = 0. 

n—>-oo m 

Let e > 0. Going back again to the setting of Section II-B, 
we encode the confidential DMS using a traditional source 
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code as in Lemma 1, and the public DMS using a uni¬ 
form compression code as in Proposition 2. The correspond¬ 
ing source encoder-decoder pairs are denoted {fn,9n) 
ifP,gP), respectively, and we set = /^(K") G 
and Mp = G |l,2"'''^p]. We assume n large 

enough so that 


(e 


z'^ mcTTlp 

X bMp(mp) -Pc/„= (wp; 


P[(K”,^p”) b (ff=(Me),5^(Mp,C/dJ)] ^ e, 




< e. 


( 6 ) 

(7) 


— 2e ^ 
(/) 

< 3e, 




( 11 ) 


Under the conditions (3)-(5) of Lemma 2, which are met 
whenever 

H{V,) + H{Vp) < I{X-Y\Q), 

H{V,)<I{X-Y\Q)-I{X-Z\Q), 

H{Vp)>I{X-Z\Q), 

for n sufficiently large there exists a wiretap code C„ with 
encoder/decoder pair {fn,9n) so that for any rric, and for Mp 
distributed according to P( 7 „h > the uniform distribution over 





^ P 

Mp b Mp Me = TOc 

< e, 

(8) 

= P 

Me b Me Me = TOc 

< e, 

(9) 

= P 

V {pZ”'\Mp=mp,PZ”^] 

/A 

JT, 

(10) 

< 2e, 


where (a), (c), and (e) follow by the triangle inequality, (6) 
holds by (10), (d) and (/) hold by (7). 

Consider then an optimal coupling [46] between Mp and 
Mp such that P[£’] = V(pMp,P[/„jjp), where £ = {Mp ^ 
Mp}. We have for any TOc, 


Mp ^ Mp|Mc = TO, 

Mp^Mp|M, = TOe,£"JP[£’"] 

-I-P Mp Mp\Mc = rnc,£ P ] 

Mp ^ Mp|Mc = mc,£ 

Mp ^ Mp|Mc = TOc, £ 


¥[£] 

'^{pm^,Pu^rJ 


where {Mp,Mc) is the estimate of {Mp,Mc) by the decoder 
of C„, and for any z”, TOc, TOp, 


Pz^M^Mpiz^jmcmp) 
A 


= Pz^\M^=m^,Mp=mAz'^)pMArnc)pu„R^ {mp). 


Mp^ Mp\M^ = mc + V(pm„,P;7„pJ 


where the last inequality follows from (7) and (8). Similarly, 
using (7) and (9), we have for any TOc, 


Me b Mc\XIc = rric 


^ 2e. 


Note that (8)-(10) holds by Lemma 2 because we have 
assumed Mp uniformly distributed. We now study the con¬ 
sequences of using the wiretap code C„ with Mp (not exactly 
uniformly distributed) instead of Mp. Specihcally, we note 
(Mp,Mc) the resulting estimate of (Mp,Mc) by the decoder 
of Cn, and dehne for any z”, rric, mp, 

Pz^M,Mp{z’^,mc,mp) 

— PZ"\Ma=ma,Mp=mp{z )PMa{mc)pMp{mp). 

We then have for any rric, 

V {pZ'^\Ma=mp,PZ") 

(a) 

^ V {pZ'^\Mp=mp,PZ'^\Mp=mp) +V (pZ" |M<,=mc > PZ" ) 

+ Y {pzp,PZ--) 

(b) 

< e -f V {pz^\Ma=7np^PZ^\Ma=mp) + ^ {PZ^,PZ'^) 

(* 7 ) 

^ ^ ^ ^ ^ ' {PZ”-\Mp=mp,Mp=mp{z ) 

2^ mp 

x|pMp(mp) -p[/„^^(TOp)|) +V(pz-,PZ") 

= e + V (pMp,PUr,Rp') {pz^,Pz-) 

(d) 

^ 2e + V(pz^,pz") 


Encoding the sources into codewords with Cn as 
/n(/^(K”)i/n(17p”,Ud„)), and forming estimates from the 
channel output F” as = g^{gn(Y'^)), and V}" = 

gP{gn{Y^),UdJ, we obtain again 


(KM/") b (v;m>p")_ 

^ P [(K", K”) 7 ^ (K", V}”)l(^p, = (Mp, M, 

+ p[(Mp,M,) + {Mp,Mc) 

sG 5e. 


For any u" G V", we also have 

(a) ^ 


m 

^ 3e, 


where (a) follows by the triangle inequality, [h] holds because 
Z" —> Me —> Vc- Since e > 0 can be chosen arbitrarily small, 
we obtain again the achievability part of Theorem 1 . 























V. Conclusion 


We have proposed and analyzed two coding architectures 
for multiplexing confidential and public messages and achieve 
information-theoretic secrecy over the wiretap channel. Our 
first architecture relies on wiretap codes that do not require 
uniform randomization, while the second architecture exploits 
compression codes that output nearly uniform messages. By 
showing that secrecy can be achieved with only vanishing-rate 
randomness resources, and without reducing the overall rate 
of reliable communication, the proposed architectures establish 
that secrecy can be achieved at negligible cost. 

An important issue that we have not addressed is the design 
of universal wiretap codes that merely require that the public 
message carries enough randomness, and do not require the 
knowledge of the statistics. Some results in this direction are 
already available in [38]. Finally, the design of actual codes 
for the proposed architecture remains an important avenue for 
future research. 

Appendix A 

Converse of Theorem 1 

We consider the problem described in Section II-B when 
the uniformly distributed seed is shared between the encoder 
and the decoder, as it is the case in Section IV. Obviously, the 
converse will also hold when the seed is not available at the de¬ 
coder, as it is the case in Section III. We develop our converse 
following techniques similar to Csiszar and Korner [23] and 
Oohama and Watanabe [37]. Although the ideas are similar, 
the converse does not follow directly from these known results 
because of the presence of a seed with length d„. Formally, 
consider two sources (VcPVo) {Vp,pvj,) that can be 
transmitted reliably and secretly. Then, there exists a code with 
block length n such that 

Vp ^ (K", Vp) < e; (reliability), (12) 

(secrecy), (13) 

dn/n ^ Pn (sub-linear seed rate), (14) 

where lim„_>oo e'„ = = lim„_>oo Pn = 0. We also 

define -f 1 /n. Consequently, 

Hivp H{V^Vp - H{Vp 

= I{V^Vp,Y^UP + H{V^VpY^UdJ - H{Vp 

2 Y^UdJ + nen - H{Vp 

^ I{v^vp, Y^UdJ + nen - I{V^; Z^\VP 

2 nv^vp, Y^UdJ + ne„ - + nSn 

< YP- I{V^VP, ZP + ne„ + nS^ + 

(15) 

where (a) holds by the independence of the sources, (6) holds 
by (12) and Fano’s inequality, (c) holds by (13). Next, 

H(vp + npn > H{vp + dn 

HiVp^Upvp 


(c) 

> HiX^\VP 

^ iix^-,z^\vp 

$ I{X'--Z^)-n5n, (16) 

where (a) and (5) hold by independence of the sources and 
the seed, (c) holds because X^ is a function of V^, Ud^, V^, 
(d) holds by (13), (e) holds because — X^ — Z" forms a 
Markov chain. Similarly, 

HiVP + H{VP HiVp^VpUdJ 

= I{V^VP, V"|C/dJ + Hp^VpY'-Ud^) 

(17) 

where (a) holds by independence of the sources and the seed, 
(6) holds by (12) and Fano’s inequality. Finally, 

(o) 

npn > dn 

= HiUdJ 
^ HiUdJVJ^Vp 

H{X^\V^Vp 

^(18) 

where (a) holds by (14), (6) holds by uniformity of the seed, 
(c) holds because X” is a function of V^, Ud „, 14". The single 
letterization is obtained by introducing a random variable / 
uniformly distributed over |l,n] and defining 

Q, = {Yr\zp,), Y = {Q^, v:, Vp, 
Q={Qj,I), V = (Vf,/), 

X = Xi, Y = Yi, Z = Zi. 

Note that the joint distribution of Q, V, X, Y, Z factorizes as 
PQPv\qPx\vWyz\x- Then, using Csiszar’s sum-equality 

I{V^Vp, F”) - Z") 

n 

^ E y^\Yr") - i{VpVp z^zp)] 

n 

= E Y^Yp^zp) - ip^vp, Zppzp)] 

= n[I{V-,Y\Q)-I{V-Z\Q)]. (19) 

In addition, 

/(X”;Z"|V4Vp") 

n 

= E [H{Z,\Z{+P^Vp - H{Zp{+^X^V^Vp] 

n 

^Y,[H{Z^\Yp^ZpV^Vp-H{ZppYp^X,V^V;] 

n 

i=l 
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= nI{X;Z\V), (20) 

where the inequality holds because Zi — Xi — 
forms a Markov chain. Similarly, 

n 

n 

= ^ {H{Z,\Z^+,Yl-^) - H{Z,\X,Z^+,Yr^)) 

i^l 

n 

= ^/(X,;Z,|Q,) 

= nI{X-,Z\Q). (21) 

Finally, 

I{V^V:Ud^-Y^) 

n 

= Y.^KV:Ua^-Y,\Yl-^) 

n 

^ ^/(y;K"(/d„yrizr+i;y,) 

2^1 

n 

2^1 

n 

= Y,I{X^Q^■,Yi) 

2 = 1 

= nI{XQ-Y) 

= nI{X-Y\Q), (22) 

where the second equality holds because Xi is a function 
of V^,Ud^,V^ and F, - Z'^^^Yl-^X, - V^V^Ud^ forms 
a Markov chain. Combining (15) - (22) we obtain 

H{V,) ^ I{V- Y\Q) - I{V- Z\Q) + en + 5n+^Jin 
H(yp) + ^ir,^I{X■Z\Q) 

H{V,) + H{Vp) ^ I{X- Y\Q) + e„ 

I{X-Z\V). 

Note that using pqvxyz = PqPv\qPx\vWpyz\x we have 

I{V; Z\Q) = liVX; Z\Q) - I{X; Z\QV) 

= I{X- Z\Q) + I{V- Z\QX) - I{X- Z\V) 

^ I{X-Z\Q)-pr., 

and 

I{V;Y\Q)^IiVX-,Y\Q) 

= I{X;Y\Q)+!{¥■,Y\QX) 

= I{X;Y\Q). 

Hence, we must have 


Appendix B 

Proof of Proposition 1 

We fix a joint distribution pgx on Q x X such 
that2 IiX;Z\Q) < lim™ iiF 2 (Mp) and /(X;F|Q) - 
I{X-,Z\Q) > 0. Let e > 0, i?o > 0, and n e N. 
We randomly construct a sequence of codes {C„}„gN as 
follows. We generate 2"^“ sequences independently at random 
according to pg, which we label q^{i) for i G |1,2"^“]. 
For each sequence q^{i), we generate sequences 

independently a random according to Px\Q^ which we label 
x"‘{i,j,s) with j e |1,2"^‘=] and s G |1,2"^3’]. To transmit 
a message i G |1,2”^“] and j G |1,2”'^=], the transmitter 
obtains a realization s of the public message Mp G |1, 2"^3’], 
and transmits s) over the channel. Upon receiving y”. 

Bob decodes i as the received index if it is the unique one such 
that {q^{i),y^) G TJ^{QY)-, otherwise he declares an error. 
Bob then decodes (j, s) as the other pair of indices if it is 
the unique one such that a;"(i, j, s), t/") € TJ^{QXY). 

Similarly, upon receiving z^. Eve decodes i as the received 
index if it is the unique one such that {q^{i),z'^) G TJ^{QZ)\ 
otherwise she declares an error. For a particular code Cn, 
we note Pe(Cn) the probability that Bob does not recover 
correctly (i, j, s) and that Eve does not recover correctly i. 

Lemma 3. If Rq < I(Q\Y) and R^ + Rp < I{X\Y\Q), then 
E[Pe((7„)] ^ 2““" for some a > 0. 

Proof: The proof follows from a standard random coding 
argument and is omitted. ■ 

Lemma 4. If lim —H 2 {Mp) > I{X;Z\Q), then we have 

n—^oo Ti 

[V(pMcZ",Pm<,PZ")] ^ 2“^" for some /? > 0 and all 
n € N sufficiently large. 

Proof: The proof relies on a careful analysis and mod¬ 
ification of the “cloud-mixing” lemma [47]. Let e > 0. Eor 
clarity, we denote here pg^x^z^ the joint distribution of 
(Q", induced by the code, as opposed to pg^x^z^ 

defined as 

pg^x^zAq^,^\n=Pz^\xAz-\xnPx^QA^^,Qn- 

Eirst note that the variational distance Y{pm^z^ , Pm^Pz'^ ) can 
be bounded as follows. 

V {pm^z^ , Pm^Pz^ ) 

^ ^{pm^Q'^ z^ tPm^Pq^Z'^) 

= EgriMe [^{PZ'^\M^Q^,PZ'^\Q^)] 

^ EgriMe [npz "|M,Q",PZ'*|Q") +^{PZ’-\Q^,PZ'^\Q^)] 
^ 2EQr.jvf, [V(p2^|MeQ",PZ"|Q")] 

Then, let Q” be the sequence in Q" corresponding to Mq = 1. 
By symmetry of the random code construction, the average 
of the variational distance 'V{pM^Z’^,PMaPz^) over randomly 
generated codes Cn satisfies 


H{V,) < I{X-Y\Q) - I{X;Z\Q) + en + Sn + 2pn, 
H{Vp)^I{X-Z\Q)-pn, 

H{V,) + H{Vp) < I{X- Y\Q) + £n. 


Ec„ [y(PMaZ’',PMaPZ")] 

^If such a probability distribution does not exist the result of Lemma 1 is 
trivial and there is nothing to prove. 
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^ 2Ec„ [V(p^r.|Q^^QjM, = l,PZ"|Q- = Q 5 ‘)] , 


where 


PZ"|Q"=QrM,=i(2:”) = ^ Pz-|X"(2”|a;"(l,l,fc))pMp(A:). 
fc=i 


The average over the random codes can be split between the 
average of Q" and the random code C„(( 7 ") for a fixed value 
of g", so that 


Ec„ [V(pzri|Q„^Q^M-, = l,PZ"|Q"=Q")] 

= X! )Ec„(qr) 

^ X! P'^"(9nEc„(g^) [V(pz..|Q~^q„jvf^^l,P2’*|Q"=9")] 

<Ji"6r,"(C/) 

+ 2 P[Q"^r"(Q)], 

where the last inequality follows from the fact that the varia¬ 
tional distance is always less than 2. The first term on the right- 
hand side vanishes exponentially with n, and we now proceed 
to bound the expectation in the second term following [47]. 
First note that, for any z" G Z", 






2'^^p 


FZ"|X"(2:”|a::”(l, l,k))pMp{k) 


k=l 


= Y [Pz^\x^{z'^\x^{l,l,k))\pM^{k) 




= FZ-|Q"=g[‘(2:”)- 


We now let 1 denote the indicator function and we define 


E 

Y Px"IQ’ 

*=5^M, = i(^") - PZ’^IQ^ 

=,r(^")| 






Y ® [fz"IQ" 

=g^M, = l(z”) +PZ-IQ" 


z'^ 




= 2 

Y FX"IQ-= 




z"^r2"(Z|q") 


which vanishes exponentially fast as n goes to infinity for 
g" G T^{Q). Similarly, taking the expectation of the term 
in (25) over C'„(g”), we obtain 


E 


E 

p(2)(2") _E 

p(2)(2»)] 

_z’'GT^H^\q^) 




^ E 


Y -E 


z'^ 


= ^ E[p^„|;,„(z"|X"(l,l,l)) 

xl{(X"(l,l,l),z")^72(XZ|gn}] 

= Y Pz"X-|Q"=9r(2",a;”), 

(x^,z^)iT^^(XZ\q^) 


which vanishes exponentially fast with n. Finally, we focus 
on the expectation of the term in (24) over Cn{qi). For z" G 
T-Jl{Z\q^), Jensen’s inequality and the concavity of a; i—>■ y/x 
guarantee that 


E 


p(i)(z’^)-E [p(i)(z”) 




Var (p(i)(z”)). 


2nRp 

pH) A ^ (Z"|CC"(1, 1, k))pM, (fc) 

k^l 

X l{{x-{l,l,k),z-) G %liXZ\q^)}, 

2nRp 

p^'^Hz”) = Y F2"|A"(^”|a:”(l,l,fc))PMp(A;) 

k^l 

xt{{x-{l,l,k),zm%:{XZ\q^)}, 

so that we can upper bound X{pzr<.\Qr.=q^M^=i,Pz^\Q^=q^) 
as 


^{PZ^\Q^=q^M, = l,PZ^\Q^=q^) 

^ Y |PZ"|Q"=<?rM, = l(2:"') -pZ"|Q-=g-(z") 


^''tPPeiZlqi) 


+ E 

p(i)(z") _E 

p(i)(z")j 




+ E 

p(2)(^n) _E 

p(2)(z")] 


z^GT^H^lqi) 


(23) 

(24) 

(25) 


Taking the expectation of the term in (23) over C„(g"), we 
obtain 


In addition, 

2^Rp 

Var(p(i)(z”)) = YPMAkf^^{PzHxAzlX-il,l,k)) 

k^l 

xt{{X-{l,l,k),z^)Gr^MZ\qm 

Note that 

Var(p2„|x.(z"|X"(l,l,fc)) 

xl{(X"(l,l,A),z-)G7;:(XZ|gn}) 

Y Fx"IQ"=9f(2^") 

xl{ix-,z-)Gr^:iXZ\q^)}f 

= Y PXHQ"=q]^{x'^)PZHX^iz'^K? 

x'^:{x^ ,z'^)GT^^ {XZ\q'^) 

2-n(H{Z\X)-S(e)) 

a:":(a:",z’*)G 7 ^"(XZ|(j") 

2-niHiZ\X)+H{Z\Q)-S(e)) ^ 
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where (a) and (6) follow from the AEP; therefore, 

2nRp 

^ ^-niHiZ\X)+H{Z\Q)-Sie)) ^ 

fc=l 

^ 2-n{HiZ\X)+H{Z\Q)-S(e))+ 

and 


eiTor. We let M = $(X",{7d„), and define Pg = P[X" ^ 

f7dj, t/dj], Ue ^ V {pm.PUmJ- 

• We first determine a condition over i? to ensure 
E$ [Ue] < e. Note that Vm G Ain, 

PM{m) = = to}, 

X"' U 

hence, on average Vto € Ain, E$ [pM{m)] = 2“""^, 
which allows us to write 




E E 

z"Gr2"(Z|gf) 

^ 2nH{Z\Q)2-^{H(,Z\X)+H{Z\Q)-Sie) + 


H2{Mp) ^ 


Hence, if lim„_>.oo ^H 2 {Mp) > I{X;Z\Q) + (5(e), the sum 
vanishes as n goes to infinity, which concludes the proof. ■ 
We point out that a generalized version of Lemma 4 may 
now be found in [38]; in fact, [38, Theorem 14] develops a 
general exponential bound on the secrecy metric, and a close 
inspection of their result shows a tighter exponent involves the 
Renyi entropy of order 1 + p with p G [0,1] in place of the 
Renyi entropy of order 2. Actually, [48] shows that this is the 
best exponent with random codes. 

Using Markov’s inequality, we conclude that there ex¬ 
ists at least one code (7„ satisfying the rate inequalities in 
Lemma 3 and Lemma 4, such that Pe(C„) ^ 3 • and 
^{PM^MoZ^ ,Pm^PMoZ^ ) < 3 • 2“^". We now dehne 

Pi (to) = f\Mc^ Mc\Mc = to] , 

P 2 (to) = P Mp ^ Mp\Mc = TO , 

S{m) = V(p^,^|Me=m,^'^")■ 

Since E[Pi(Mc)] 2"“”, E[P 2 (Mc)] ^ 2"“”, and 

E[S'(Mc)] < 2“'®", we conclude with Markov’s inequality 
that for n large enough, we have 

Pi(to) < 2-“"+2, P2(to) < 2-“”+2, S'(to)<2-^”+2 

for at least a quarter of the messages to. Expurgating the code 
Cn to retain only these messages concludes the proof. 


Appendix C 

Proof of Proposition 2 

A. Achievability 

We show next that there exists a sequence of (2"^,n, 2^*") 
uniform compression codes {C„}„gN* such that H{X) is 
achievable with a seed length dn scaling as 

dn = Q{vnVn), for any {r;„}„gN with lim Vn = +oo. 

n—)-oo 

Let ei > 0, e > 0, n e N, (i„ G N, P > 0. 
Define Mn = 2”-^ and Ain — Consider a random 

mapping $ : A"” x Ud„ -G Ain, and its associated decoder 
T' : Ain X Given {m,Ud„) G Ain x Ud^, the 

decoder outputs if it is the unique sequence such that 
i" G 7}”(Ar) and i>(x",Ud„) = to; otherwise it outputs an 


E<[, [U, 
= E<[> 


g; 


E®' 


E IPM(m) - E$ [pm(to) 

E \pm ("i) - E-f Pm 


(26) 


where Vto G Ad„, Vf G |1, 2], 

PMi'^)= E ^Pix'^,u)t{<^{x'^,u) = m}, 

x^GAi u 

with Vli = T^{X) and VI 2 — After some manipula¬ 
tions similar to those used to bound (25), we bound the 
second term in (26) as 


[E |pm - E'I> Pm ("i) ^ 4|A’|e 




(27) 


with pLx = min Px{x). Then, we bound the hrst 

a:Gsupp(Px) 

term in (26) by Jensen’s inequality 


E<[> 


m 

^ (pmH"^))- (28) 

m ' 

Moreover, after additional manipulations similar to those 
used to bound (24), we obtain 

Var<j, (p^^\m)^ 

= ^ ^p(a;", u)^Var$ (l{$(a;", u) = to}) 

oor,erji(x) u 

^ E '^p{x'',uf^i.[{t{<i>{x'^,u) = m}f] 

x^eTJiix) u 

= ^ y^p(cc”, •u)^Ej, [l{<i>(a:", m) = to}] 

x^^Tr^(x) u 

(r"Gr,"JX) u 

= p(a;")^2-‘^2-’^^ 

x^eTjiix) 

E exp2[-2n(l-ei)P(A:)]2-'^" A 


€ 


^ |r”(X)|exp 2 [-2n(l - e^)H(X)] 2-'^"2-"« 


5 $ exp 2 [-n(l - iei)H{X)] 2-‘^"2 


— dn r) — nR 


(29) 































12 


Thus, by combining (28) and (29), we obtain 


E 


PmM 

m 

^ \/exp 2 [-n(l - 3ei)iJ(X)] (30) 

m 

dri 


= ^/M^exp2 

^ exp2 


-- (l-3ei)i/(X) + 

2 \ n 


2 V n 


(31) 


Hence, if i? < H{X) + ^ — SeiH{X), then asymptoti¬ 
cally E$ [Ue] ^ e by (27) and (31). 

We now derive a condition over R to ensure E$ [Pg] ^ e. 
We define Sq = {X" ^ Vl{X)}, and £:i = {3x" 

= $(X",{7) and x” e 7 ;”(X)} so that 
by the union bound, E$[Pe] ^ P[fo] + P[£i]- We have 


P[£:o] < 


(32) 


and defining P(a:",a;",u) = P[3a;" 7 ^ x", $(a;", it) = 
$(a;",it) and i” G 7^"(2f)], we have 

x'^ u 

u x^^TJliX) 

x'^^x^ 

= EE^^(^">“)E2-"'' 

x'^ u x'^^T^^iX) 

x'^^x^ 


■y—nR 


■y—nR 


We can also choose ei = for any i;„ with lini; 
+ 00 , such that 

d 


n—^oo 


AHiX) < 


< 4HiX) + iV^Vn)-\ 


B. Converse 

We first show that any achievable rate R must satisfy R ^ 
H{X). Assume that R is an achievable rate. We note M = 
We have 

nR ^ H{M) 

^I{X^-M\UdJ 

= H{X^\UdJ - HiX'^lMUdJ 

> i7(X"|17dJ-n(5(e) 

= nH{X) — n6{e), 

where (a) holds by Fano’s inequality and (b) holds by inde¬ 
pendence of X" and Ud„- 

Hence it remains to show an upper bound for the optimal 
scaling of (i„. Recall first the Berry-Esseen Theorem. 

Theorem 2 (Berry-Esseen Theorem). Let be a 

sequence of Ltd. random variables with E[Zi\ = fj, and 
E[(Zi — p)^] = > 0 and — pp] = pz < 00 . 

Let Yn = X+Z 2 -t"-+gn-nAi _ j^f p dgyiote the cumulative 
distribution function of Yn- Then, for any x G K, 


|F„(x)-<i>(x)|<^ 


(34) 


where $ is the cumulative distribution function of the standard 
normal distribution with mean zero and variance 1 and a is 
a constant that depends only on the distribution of Zi. 

Using Theorem 2, we show the following. 

Lemma 5. Let {Xi}jgN be a sequence of Ltd. random 
variables with each distributed according to px such that 


x'^ u 

^EE p(x", It) exp 2 [ni7(X)(l 3- ci)] 2 

X'^ U 

<exp 2 [n(iT(X)(l-f ei)-i?)]. (33) 

Hence, if i? > H{X) + eiE[{X), then asymptotically 
E<[,(Pg) < e by (32) and (33). 

All in all, if R is such that 

H{X) + eiiT(X) <R< H{X) -3 — - 3eiiT(X), 

n 

then asymptotically by the selection lemma (e.g. [5, Lemma 
2.2]), E$[Ue] ^ e and E$[Pe] ^ e. Thus, we choose dn such 
that 

4neiiT(X) < d„ < 4neii7(X) + 1. 


H{X) ^ -E[logpx(^i)] < 00 , 

^E[(logpx(^i) + i^W)']>0, 
p ^E[|logpx(^i) + i?WI"] < 00 . 

Then, there exists an a > 0 such that for any a > b > 0, 

2ap 


A 

da,b = 


A 

^ 00 , & - 


P[X" G Tn(a, 6)] - ($(-6) - $(-a)) 


€ 


P[X" Gr„(oo,&)]-$(-6) 




ap 


where 


Tr^{a,b) ^ Ix^ G X 


2-nH(X)-aa,/n ^ px{x^) 

■ > Pxlx^) 


-. Then, 


which means c?„ = Q{vns/n). Einally, by means of the selec¬ 
tion lemma applied to Pg and Ug, there exists a realization 
of 4) such that Up ^ e and Pg ^ e. 


Proof: Define S'„ = 

P[X" G r„(a, b)] =V[-a<Sn^ -b] 

= F[Sn^-b]-F[Sr^^-a]. 

Hence, 

Pa,b = |P[X" e r„(a,6)] - ($(-6) - $(-a))| 

< I P ^ -b] - $(-6)| + |P [Sr^ ^ -a] - $(-a)| 
(U 2ap 


^Note that we cannot make ei decrease faster because of (27) and (32). 


a'^\fn 























13 


where (a) holds by the triangle inequality, and (b) holds by 
Theorem 2. The bound on 7700,6 holds similarly. ■ 

We will also make use of the following lemma. 


Lemma 6. For any 

In 


\Je{4>n) ^ 2 


: A"" X Ud„ Ain cind for any 




2 d„- 

In _ 



Proof: Let fn ■ 'L'" xUd^ —>• A4n- We apply [49, Lemma 
2.1.2] to (j>n so that for any S N, for any a, for any T > 0 


l^eifn) 

= 2 '^ (P0„(X",(7d„),PC/M„) 

^ P[(X”, UdJ i ^;(a)] - P[C/m„ G T„(a + T)] - 
= P[X" iSn{a- djn)\ - P[17m„ G r„(a + T)] - 


where 

s'M 

= ^{x^,UdJ&X'^xUd^ 
= \{x'^,UdJ G X'-xUd^ 


1 

n 


log 


P: 




1 _ 

(a:",Ud„) 



1 , 1 ^ 

— log -;-r ^ a- 

n Pjfn(x”) n 


S„(a) 

Tn{a) = <{ u G : - log „ ^ , < a 

n Pum„ m 


For any 7 ^ e]0,M„[, we choose T = ^ log ^ and a = 
i log 7 „, such that a + T = ^ logM„ and P[C/m„ G Tn{a + 
T)] = 0. Hence, we obtain 


^ P[X" i 5„(a - d„/n)] - 


„-nT 


7 n 


Proposition 5 (Converse). Let for each n G N, C„ be 
an (2”^, 77 , 2'^") uniform compression code Cn for a DMS 
{X,px) such that 

lim P^{(j)n,fn) = lim iJ^{(j)n) = 0 . 

n—^oo n—>-oo 

Then, d„ = n{y/n). 

Proof: Since the encoding function </)„ of C„ utilizes a 
seed Ud„ taking values in | 1 , 2 ‘^"] that is independent of the 
source 2 f”, we can find such that 

P[X" ^ <J, <J] ^ Pe(</'n, ^n). (35) 

Fix a > b > 0, and define Cn{a, b) as 

Cn{a,b) = {x" G Tn{a,b) : x” = ipnifnix^, u*dj, u*dj}. 


Note that 


P[X" e£„(a,6)] 

P[X" e r„(a,6)] -P[X" ^ 7/;„(,/>„(X",773J,7r3J] 

§V[X" er„(a,6)]-Pe(./>„,7/>„) 

1^) 2o:/7 

^ T>(-6) - T>(-a)- - Peifny'fn) 

= Vn(a, b), 

where (o) follows from (35), and (b) holds by Lemma 5 
with a^, p defined therein. Note that for any a;" G Cnifl^b), 
px{x^) < Hence, 

2JH(i)+6<!!/s ^ ^ ^ 

Since £„ (a, b) is a subset of source realizations for which 
the code offers perfect reconstruction (when the seed used is 

Ud„ = Wd„)’ 

M„ ^ |£„(a, &)|^ Vn{a, b) ( 35 ) 

We now use Lemma 6 with 

7„ ^ 7;n(a,6) 2"^(^), M„ ^ r;„(a,6) 


which yields 






Ue(^n) 


2 |X„| 

^ "^* 6 (^ 71 ) I rf — ba^/n 


(37) 


From Lemma 5, it follows that 




Vn{a,b) 2”-f^(^)_ 


log: 


7 ( 0 ,&) 


4) 

a^n 

Combining (37) and (38), we obtain 


ap 


(38) 


$ 


log; 


i(a,fc) 


crVri 


\ A 1 Ue(^/^n) r, — b( 7 ^/n 

Pn — r z r- 

2 cr'’\/r 


Rearranging terms and taking appropriate limit, we get 


lim 


n—>-oo aWn 


= 

= $-1 


$ lim 

yn—)-oo Gyn 

lim $ ( 

n—700 \ aPn 


( lim /3„) = $ ^( 1 ) = 00 , 

\n—>^00 / 


where in the above arguments, we have used the fact that $ 
is invertible, continuous and increasing. ■ 


Remark 5. In [2], we prove a converse for i.i.d. sources that 
is stronger than Proposition 5. If dn = o{y/n), then 

limsup Pe( 0 „, 7 /j„)+limsup Ue((/)„) ^ 1 . 

n—^oo n—>-oo 

We however do not need this stronger statement for our 
purpose. 
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Appendix D 

Proof of Proposition 3 

Let e > 0, (5 > 0 and n S N. Let t, m, and dn to be 
expressed later. We know from [43], [44] that there exists 
an invertible (to, d, to, e)-extractor EXTq, such that (2) is 
satisfied. Assume that the emitter and the receiver share a 
sequence Ud„ of d„ uniformly distributed bits. As described 
in Figure 5, we proceed in two steps to encode X". First, 
we perform a compression of to form S based on eo- 
letter typical sequences, eo > 0 , we note this operation 
4>'n '■ -L” —>• such that S = and we note 

^>(1 : > A”” the inverse operation such that 


= 2neoH{X) + log 


1 + 


Kin) 

1 - 6eoin)_ 


+ 2 log[u.(l €q)H (Ai)] + 2 log —h 0 ( 1 ). 


Then, we choose cq = for any Vn with lim„_>.oo Vn = 
+ 00 , such that 



< 2H{X) + 


2 1 

-^ log - + O 

VnVn e 



which means dn = 0{vn\/n). 


lim P[X"^Co^;(^")] = 0. (39) 

n—>oo 

Note that this compression implies 

limsup - log|Al(i|< id(2f) + (5. (40) 

n—>-00 ^ 

Then, we apply the extractor FXTq to S and Ud„, to form 
the encoded message M = FXTo(S', 17d„). We define the 
encoding function (j)n ■ A”" x lAd„ —)■ as 


Appendix F 

Proof of Proposition 4 

Let /3 ejO, l/2[. Let n e N and N = 2". We set = 
X^Gn- We define the following sets. 

Vx={^e [1,W1 >l-d^}, 

Hx={tGll,Nj:H{A,\A^-^)>6N}. 


((.„(X”, UdJ^M = EXTo«(X"), UdJ, 

and the decoding function ipn - Ain x Ud„ —>■ Af" as 

MM. UdJ ^ Kni^XT^iM, UdJ) 

= KniS) 

= KnOKiXn, (41) 


These sets cardinalities satisfy the following properties. 
Lemma 7. The sets TLx o-nd Vx satisfy 

1) limx^+K'Hxl/N = HiX), 

2) limjv-). +KK\/N = HiX), 

3) limjv^+oo|Hx\Vjf|/W = 0. 


which is possible since EXTq is invertible. Note that by (39), 
(41), we have 

lim ¥[X^ ^ijnifniX^KdJKdJ] 

n—foo 

= lim ¥[X^^KnoKiXn] 

n—>-oo 

= 0 , 

and since the sizes of the first input and output of the extractor 
are the same, by (40), we have 

lim sup - log|A4(j|< HiX) + S. 

n—foo ^ 

Moreover, [43], [44] also shows that Ue ^ e. It remains to 
show that for any eb > 0 , we can choose d„ = 0 (n^/^+'^'’). 
Let eo > 0- As in the proof of Lemma 1, we may show 


Proof: 1) follows from [45] and [50]. 2) follows from 
[51, Lemma 1] which also uses [50]. 3) holds by 1) and 2) 
since Vx C TLx- ■ 

Lemma 8. The output of the encoder A^[Vx] is near 
uniformly distributed with respect to the Kullback-Leibler 
divergence. 

Proof: We have 

H{A^[Vx]) = E H{AU^-^[Vx]) 

i&Vx 

^ Y, HiA,\A^Y 

iGVx 

^ ll^xKl — Sx ), 


HooiS) = -log(maxps(s)) 

> n(l - eo)H{X) - log 

We define 

t = n{l- eo)H{X) - log 


Seojn) 

I-Kin) 


SKn) 

I-Kin) 


(42) 


Thus, since the input size to of the extractor verifies to ^ 
[n(l + eQ)H{X)~\, by (2) and (42) we obtain 

d„ ^ n(l + eo)H{X) - t + 2 log[n(l + €o)H (X)] 

+ 21 og - + 0 ( 1 ) 
e 


where the first inequality holds because conditioning reduces 
entropy and the last inequality follows from the definition of 
Vx- We thus obtain 

log 2 lv-l - H{A^[Vx]) < ^ NSx- 


Finally, by [45], the receiver can reconstruct X^ from 
A^\Vx] and Ig = A^[Hx\Vx], where Ig is encrypted via a 
one-time pad with the uniform seed shared by Alice and Bob. 
Hence, by Lemmas 7, 8 , we obtain a polar code construction 
for a uniform compression code, whose seed length scales 
as o{N). 
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